In [31]:
%load_ext autoreload
%autoreload 2

The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload
In [32]:
import pandas as pd
import plotly.io as pio
from idea.profile.profile import calculate_profile
from idea.validation.validation import validate_roadwork

from util.visualization import create_multiplot_heatmap, create_plot_by_segment

pd.options.plotting.backend = "plotly"
pio.renderers.default = "notebook"

Example Validations¶

This repository contains three example cases:

  • Example 1: There is always the maximum amount of FCD coverage during the profile period. Even during the roadworks, coverage does not decrease. → Status: open

  • Example 2: There is always the maximum amount of FCD coverage during the profile period. During the roadworks, there is no coverage, but occasionally a vehicle is detected. → Status: closed during periods with less than usual coverage

  • Example 3: There is consistently very low FCD coverage during the profile period. During the roadworks, coverage drops to none, with occasional vehicle detection. There is one short period where coverage increases, and the algorithm marks it as open. → Status: closed, with one short open period due to increased coverage

In [33]:
# Choose the test you want to run (1 till 3 available)
EXAMPLE_FOLDER = "example2"
In [34]:
# Retrieve fcd during profile period.
df_minute = pd.read_csv(rf"resources/{EXAMPLE_FOLDER}/fcd_profile_input.csv", sep=";", index_col=0)
df_minute.index = pd.to_datetime(df_minute.index, utc=True)
start = pd.Timestamp("2024-01-01 00:00:00", tz="UTC")
end = pd.Timestamp("2025-01-01 00:00:00", tz="UTC")
profile = calculate_profile(df_minute, start, end)

# Retrieve fcd during roadwork.
validation_start = pd.Timestamp("2025-03-15 12:00:00", tz="UTC")
validation_end = pd.Timestamp("2025-03-15 15:00:00", tz="UTC")
fcd_during_roadwork = pd.read_csv(
    rf"resources/{EXAMPLE_FOLDER}/fcd_during_roadwork.csv", sep=";", index_col=0
)
fcd_during_roadwork.index = pd.to_datetime(fcd_during_roadwork.index, utc=True)
In [35]:
validated_roadwork = validate_roadwork(fcd_during_roadwork, profile)
In [36]:
validated_roadwork
Out[36]:
time fcd consecutive_zeros consecutive_low day_of_week hour_of_day max_consecutive_zeros_q95 max_consecutive_zeros_or_ones_q95 fcd_mean_median running_mean segment_closure_status
0 2024-01-05 00:00:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.500000 undetermined
1 2024-01-05 00:01:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.166667 undetermined
2 2024-01-05 00:02:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.055556 open
3 2024-01-05 00:03:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.018519 open
4 2024-01-05 00:04:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.006173 open
... ... ... ... ... ... ... ... ... ... ... ...
14395 2024-01-14 23:55:00+00:00 10 0 0 Sunday 23 0.0 0.0 10.0 0.000000 open
14396 2024-01-14 23:56:00+00:00 10 0 0 Sunday 23 0.0 0.0 10.0 0.000000 open
14397 2024-01-14 23:57:00+00:00 10 0 0 Sunday 23 0.0 0.0 10.0 0.000000 open
14398 2024-01-14 23:58:00+00:00 10 0 0 Sunday 23 0.0 0.0 10.0 0.000000 open
14399 2024-01-14 23:59:00+00:00 10 0 0 Sunday 23 0.0 0.0 10.0 0.000000 open

14400 rows × 11 columns

In [37]:
# Add order column (needed for plot function)
validated_roadwork["order"] = 1
In [38]:
validated_roadwork.head()
Out[38]:
time fcd consecutive_zeros consecutive_low day_of_week hour_of_day max_consecutive_zeros_q95 max_consecutive_zeros_or_ones_q95 fcd_mean_median running_mean segment_closure_status order
0 2024-01-05 00:00:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.500000 undetermined 1
1 2024-01-05 00:01:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.166667 undetermined 1
2 2024-01-05 00:02:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.055556 open 1
3 2024-01-05 00:03:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.018519 open 1
4 2024-01-05 00:04:00+00:00 10 0 0 Friday 0 0.0 0.0 10.0 0.006173 open 1

Visualization of the validation values for a single segment.¶

In [39]:
fig = create_multiplot_heatmap(
    validated_roadwork,
    [
        "fcd",
        "running_mean",
        "segment_closure_status",
        "fcd_mean_median",
        "max_consecutive_zeros_q95",
        "max_consecutive_zeros_or_ones_q95",
    ],
)
fig.show()

Visualization of the running mean for a single segment¶

In [40]:
fig = create_plot_by_segment(validated_roadwork)
fig.show()